Probabilistic Inference: Test and Multiple Tests

In this paper, we view that real-world scientific inference about an assertion of interest on unknown quantities is to produce a probability triplet (p, q, r), conditioned on available data. The probabilities p and q are for and against the truth of the assertion, whereas r = 1 − p − q is the remaining probability called the probability of “don’t know”. Such a (p, q, r)-formulation provides a promising way of representing realistic uncertainty assessment in statistical inference. With a brief discussion of what we call inferential models for producing (p, q, r) probability triplets for assertions, we focus on a particular inferential model for inference about an unobserved sorted uniform sample. We show how this inferential model can be used for (i) single tests, (ii) robust estimation of the empirical null distribution in the context of the local FDR method of Bradley Efron, and (iii) largescale simultaneous hypothesis problems, including the many-normal-means problem and the problem of identifying significantly expressed genes in microarray data analysis. These examples indicate that hypothesis testing problems can be formulated and solved in a new way of probabilistic inference.